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Abstract. We obtain the variational equations for backward stochastic differential equations in recursive 
stochastic optimal control problems, and then get the maximum principle which is novel. The control domain 
need not be convex, and the generator of the backward stochastic differential equation can contain z. 
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1 Introduction 


Let (17,7^, P) be a probability space and let W be a d-dimensional Brownian motion. The filtration {Tt : 
t > 0} is generated by W, i.e., 

Pt := cr{W(s) : s < t} V Af, 


where A/” is all P-null sets. Let P be a set in and P > 0 be a given terminal time. Set 

■= {(m(s))s6[o.t] : u is progressively measurable, u(s) € U and E[ / |u(s)|^ds] < oo for all /3 > 0}, 

Jo 


where U is called the control domain and U[Q,T] is called the set of all admissible controls. In fact, we just 
need P[U |M(s)|^°ds] < oo for some /3o > 0. For simplicity, we do not explicitly give this /3o in this paper. 
Consider the following state equation: 


{ dx{t) = 6(t, x(t), u{t))dt + a{t, x{t),u{t))dW{t), 
a:(0) = xq G R", 


( 1 . 1 ) 


where b : [0, T] x R" x R'^ —>• R", a : [0, T] x R” x R'^ —>• The cost functional is defined by 

J{u{-)) = E[<j){x{T) + f f{t,x{t),u{t))dt], (1.2) 

do 
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where </> : R." —>■ R, 5 : [0, T] x R" x R^' —?> R. The classical stochastic optimal control problem is to minimize 
J{u{-)) over U[0,T]. If there exists a it G U[0,T] such that 

J(u(-))= inf J(m(-)), 

ii^ZA [0,T^] 

u is called an optimal control. 2 ;(-), which is the solution of state equation (HH) corresponding to n, is called 
an optimal trajectory. The maximum principle is to find the necessary condition for the optimal control u. 

The method for deriving the maximum principle is the variational principle. When U is not convex, we 
use the spike variation method. More precisely, let £ > 0 and C [0,T] with \Eg\ = e, define 


u%t) = u{t)lE^{t) +ulE,{t), 


where u G U. This is called a spike variation of the optimal control u. For deriving the maximum 
principle, we only need to use Eg = [s, s + £] for s £ [0,T — e] and e > 0. The difficulty of the classical 
stochastic optimal control problem is the variational equation for x(-), which is completely different from 
the deterministic optimal control problem. Peng [l^ first considered the second-order term in the Taylor 
expansion of the variation and obtained the maximum principle for the classical stochastic optimal control 
problem. 

Consider the following backward stochastic differential equation (BSDE for short): 

f{s,x{s),y{s),z{s),u{s))ds - z{s)dW{s), (1.3) 


y{t) = <l){x{T)) 


where (/) : R" —>■ R, / : [0, T] x R” x R x R“ x R'' —R. Pardoux and Peng [ll| first obtained that the BSDE 
KOI) has a unique solution {y{-),z{-)). DufHe and Epstein introduced the notion of recursive utilities 


in continuous time, which is a kind of BSDE with / independent of z. In 


, the authors extended the 


recursive utility to the case where / contains z. The term z can be interpreted as an ambiguity aversion 
term in the market (see [l|). 

When / is independent of (y, z), it is easy to check that y(0) = E[(j){x(T)) + f{t,x{t),u(t))dt]. So 

it is natural to extend the classical stochastic optimal control problem to the recursive case. Consider the 
control system which contains equations dm and (ESI. Define the cost functional 


J(u(-)) = y(0). 


(1.4) 


The recursive stochastic optimal control problem is to minimize J(u(-)) in p.4l) over U\Q, T]. When the 


ME 


control domain U is convex, the local maximum principle for this problem can be found in [2l.l7l.ll3Lll7l.ll9ll21| 
and the references therein. In this paper, the control domain U is not necessarily convex, we must obtain 
the global maximum principle by the spike variation method. 

One direct method for treating this problem is to consider the second-order terms in the Taylor expansion 
of the variation for the BSDE as in [i^ . When / depends nonlinearly on z, there are two major 

difficulties (see |24l|b (i) What is the second-order variational equation for the BSDE (11.311 . which is not 
the one similar in [l2|. (ii) How to get the second-order adjoint equation which seems to be unexpectedly 
complicated due to the quadratic form with respect to the variation of z. 

Based on these difficulties, Peng [l^ proposed the following open problem in page 269: 
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“The corresponding ‘global maximum principle’ for the case where / depends nonlinearly on z is open, 
except for some special case.” 

Recently, a new method for treating this problem is to see z(-) as a control and the terminal condition 
y{T) = (j}{x(T)) as a constraint, then use the Ekeland variational principle to obtain the maximum principle. 
This idea was used in [ 9 , 101 for studying the backward linear-quadratic optimal control problem, and then 
was used in 


2C 


24j for studying the recursive stochastic optimal control problem. But the maximum principle 


contains unknown parameters. 

In this paper, we overcome the two major difficulties in the above direct method. The second-order 
variational equation for the BSDE (11.31) and the maximum principle have been obtained. The main difference 


of the variational equations with the ones in [1^ lies in the term {p{t),Sa{t))lE^{t) (see equation (13.11) in 
Section 3 for the definition of p{t)) in the variation of z, which is 0{e) for any order expansion of /. So 
it is not helpful to use the second-order Taylor expansion for treating this term. Moreover, we also obtain 
the structure of the variation for {y, z) and the variation for x. Based on this, we can get the second-order 
adjoint equation. Due to the term (p(t), 6a(t))lE^ (t) in the variation of z, our global maximum principle is 
novel and different from the one in 20l. l24|. which c omp letely solves Peng’s open problem. Furthermore, our 


maximum principle is stronger than the one in 20|, [2^ (see Example 13.81) . 


This paper is organized as follows. In Section 2, we give some basic results and the idea for the variation 
of BSDE. The variational equations for BSDE and the maximum principle have been obtained in Section 3. 
In Section 4, we obtain the maximum principle for the control system with state constraint. 


2 Preliminaries and idea for variation of BSDE 


The results of this part can be found in 
need the following assumption: 
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23 . For the simplicity of presentation, we suppose d = 1. We 


(Al) b, a are twice continuously differentiable with respect to x] b, b^, b^x, o’, cFx, ctxx are continous in 
{x, u)] bx, bxx, cTx, cTxx are bounded; b, a are bounded by (7(1 -I- \x\ -I- |u|). 

Let u{-) be the optimal control for the cost function defined in (na) and let x{-) be the corresponding 
solution of equation dm- Similarly, we define {x^{■), {■)). Set 


6(.) = (6i(-),...,&"(-)r, a(.) = (oi(-),...,a"(-)r, 

b{t) = b{t,x(t),u{t)), Sb{t) = b{t,x(t),u) — b{t), 


( 2 . 1 ) 


similar for bx{t), bxx{t), Sbx{t), 6bx^{t), ct (<), ax{t), al.x{t), 6a{t), 5(7x{t) and Sa^xit), i < n, where bx = 
{bx.)ij. Let Xi{-), i = 1,2, be the solution of the following stochastic differential equations (SDEs for short): 


{ dxi{t) = bx{t)xi{t)dt + {ax{t)xi{t) -\- 5a{t)lEAi)}dW{t), 

xi(0) = 0, 


( 2 . 2 ) 
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dx2{t) = {bx{t)x2{t) + 6 b{t)lEAt) + ^bj;j;{t)xi{t)xi{t)}dt 

' +Wxit)x2it) + 6aa;it)xi{t)lE,it) + ^(^xxit)xi{t)xi{t)}dW{t), (2-3) 

0 : 2 ( 0 ) = 0, 

where bxx{t)xi{t)xi{t) = {tr[bl.^{t)xi{t)xi{t)'^],... ,tr[b^^{t)xi{t)xi{t)'^])'^ and similar for crxx{t)xi{t)xi{t). 


Theorem 2.1 Suppose (Al) holds. Then, for any /3 > 1, 

E[ sup \x^{t)-xit)n=Oie^), (2.4) 

te[o,T] 

E[sup \x,it)\^^] = 0{eP), (2.5) 

te[o.T] 

£;[ sup 1X2(01"^] = 0 (^"^), ( 2 . 6 ) 

te[o,T] 

ii) sup |x^(f) — o;(f) — o:i(f)|^^] = 0(e^^), (2.7) 

ie[0.T] 

if) sup |a:^(f) — a;(f) — o:i(f) — a; 2 (i)|^^] = o(e^^). (2.8) 

te[o,T] 

Moreover, we have the following expansion: 


E[cf[x-{T))]- E[cl,{x{T))] = E[{cfx.{x{T)),x^{T)+X 2 {T))] 

+E[\{(t>x;x{x{T))xi{T),Xi{T))] + o(e). 


From this result, we can simply write a:i(.) = 0{^e), X 2 {-) = 0{e) and 


(2.9) 


o;®(.) = a;(.) + 0:1 (•) + a; 2 (-) + o(e). (2.10) 

This equation is the variational equation for SDE (dH) obtained by Peng in [l^ . Furthermore, he proved 
that 

E[{Md:{T)),x,{T))] = 0{e). 

This is supprise, because o;i(T) = 0{^/e). The reason is that the orderterm is in the stochastic integral 
with respect to W. Notice that 2 : in BSDE (11.31) is still in the stochastic integral with respect to W. If we 
combine the order y/e term and z, the other terms are 0(s). Maybe we can get the variational equation. In 
Section 3, we will give a rigorous proof of this idea. 


3 Variational equation for BSDE and maximum principle 

3.1 Peng’s open problem 

Suppose d = 1 for the simplicity of presentation. The results for d > 1 will be given in the next subsection. 

We consider the control system: SDE (11.111 and BSDE (11.311 . The cost function J(u(-)) is defined in (11.411 . 
The control problem is to minimize J(u(-)) over IA[Q,T\. 

We need the following assumption: 
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(A2) /, (p are twice continuously differentiable with respect to {x,y,z)-, /, Df, D^f are continous in 
{x,y,z,u); Df, D^f, cp^x are bounded; (px are bounded by (7(1 + |a;|). 


Df is the gradient of / with respect to {x, y, z), D^f is the Hessian matrix of / with respect to {x, y, z). 
Let u(-) be the optimal control and let {x{-),y{-), z{-)) be the corresponding solution of equations (11.111 
and (ll.3|) . Similarly, we define (a:®(-), 

In order to obtain the variational equation for BSDE (I1.3L we introduce the following adjoint equations: 
j -dp{t) = {fy{t)p{t) + [f^{t)a]P{t)+bl{t)]p{t) + f^{t)q{t)+a]P{t)q{t)+f,,{t)}dt-q{t)dW{t), 

< (3-1) 

p{T) = p,{x{T)), 


/ 

-dP{t) = {fy{t)P{t) + [f^{t)al{t) pbl{t)\P{f) + P{t)[f^{t)ax{t) Pbxit)] P al{t)P{f)ax{f) 
+fz{t)Q{t) + + bl^{t)p{t) + crj,,(t)[/z(t)p(t) + q{t)] 

(3.2) 

+ [Inxn,p{t),Cr]P{t)p{t)+q{t)]D^f{t)[Inxn,Pit),(T'^{t)p{t)+q{t)f}dt-Q{t)dW{t), 

P{T)= (PU^T)), 

\ 

where 

n 

fx{t) = fx{t,x{t),y{t),z{t),u{t)), p{t) = {p^{t),...,p^{t)f, bl,^{t)p{t) ='^p\t)bl^{t), (3.3) 

similar for fy{t), fz{t), D'^f{t) and cF]P,j,{t)[fz{t)p{t) +(;(<)]. By the assumptions (Al) and (A2), it is easy to 
check that the above BSDEs have unique solutions (p(-),q(-)) and (P(-),Q(-)) respectively. 

Applying Ito’s formula to {p{f),xi{t) + x^it)) + \{P{t)xi{t),xi{t)), we can get the following lemma by 
simple calculation. 

Lemma 3.1 We have 

{p{T),xi{T) +X 2 (r)) + ^{P{T)xi{T),xi{T)) = {p{t),xi{f) P X 2 {f)) P \{P{t)xi{t),xi{t)) 

+ {[{p{s),5Ks)) + {q{s),Sa{s)) P ^{P{s)6a{s),6a{s))]lEAs) 

-{fx{s) P fy{s)p{s) P fz{s)[a'^{s)p{s) P q{s)],xi{s) P X 2 {s)) 

-^{{fyis)Pis) P fz{s)[a^,^{s)p{s) P Pis)ax:{s) P a'^{s)Pis) + (3(s)] 

+ [4xn,p(s),crJ(s)p(s) P q{s)]D^f{s)[Inxn,p{s),cr'^{s)p{s) P q{s)]'^}xiis),xi{s))}ds (3-4) 

+ I^{{pis),da{s))lEAs) + {crlis)p{s) P q{s),xi{s) P X 2 is)) 

+(<5ctJ(s)p(s) + 4p(s)(5cr(s) + ^P'^{s)Sa{s), Xi{s))IeAs) 

+ Ul’^xx{s)p{s) P P{s)ax{s) P a'^{s)P{s) P Q{s)]xi{s),xi{s))}dW{s) 

+ fi^(dcr^(s)q(s) P ^[u^(s)P(s) P (7^(s)P'^(s) P Q(s) P Q'^(s)jScr(s),xi(s)}lEAs)ds. 
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By Theorem 12.11 it is easy to check that the last line of equation (13.41) is o{e) and 


<^(x"(r)) = </>(x(T)) + {p{T),x,{T)+X 2 {T)) + i(P(T)xi(T),xi(T)) +o(e). 

We set 

vHt) = y^t) - + X 2 {t)) + l{P{t)xi{t),xi{t))], 

z^t) = z^t) - {{p{t),5a{t))lE,{t) + {al{t)p(t) + q{t),xi{t) + X2{t)) 

+ + \P{t)5a{t) + \P'^ {t)5a{t),xi{t))lEAi) 

+ h{Wlx{t)p{t) + Pii)<^At) + + Q{t)\xi{i),xi{t))}- 

Remark 3.2 It is important to note that the term {p{t), Sa(t))lE^(t) in the z‘^{t) comes from the Ito’s 
formula for {p{t),xi{t)). Note that {{p{t),6a(t))lE^it)y = ({p(t),da{t))ylE^it) for any integer i > 1, so it 
is not helpful to use the second-order Taylor expansion for treating this term, which is completely different 
from the terms xi{t), X 2 {t), xi{t)lE^{t) and xi{t){xi{t))'^. In the following, we will show that this term is 
indeed in the variation of z, which is important for getting the variational equations for (y,z). 

Then by Lemma l3. 11 we can get 

y^{t) = 4){x{T)) + o{£) + {[{p{s),5h{s)) + {q{s),6a{s)) + ^{P{s)Sa{s),6a{s))]lEAs) 

+fis,x^{s),y^{s),z^{s),u%s)) - (/:r(s) + fy{s)p{s) +/ 2 (s)[ctJ(s)p(s) -\-q{s)],xi{s) + 0 : 2 ( 5 )) 
-^{{fyis)P{s)-^fz{s)[a^,,{s)p{s)+P{s)a,,{s)+a^{s)Pis)+Q{s)] 

+ [Inxn,pis),a'^{s)p{s) +q{s)]D'^f{s)[Inxn,p{s),(7'^{s)p{s) + q{s)]’^}xi{s), Xi{s))}ds 

- r{s)dw{s). 

(3.5) 

Remark 3.3 By the standard estimates of BSDEs, we can show that y‘^(t)— y(t) = 0(e), z‘^{t) — z{t) = 0(e) 
(see the proof of Theorem \S.4^ , which is the reason for constructing adjoint equations E2P and i3.2\) . 

Suppose that 

yHt) = y(t)+y(t) + o(e), 

(3.6) 

F(t) = z(t) + z(t) + o(e). 

Then from BSDE (13.51) and the Taylor expansion, we consider the following BSDE: 

yi*) = ll'{fyis)yis) + fz(s)z(s) + [(p(s),5b(s)) + (q(s),6a(s)) + ^(P(s)Sa(s), Sa(s)) 

+/(s, x(s), y(s), z(s) + {p(s), Sa(s)),u) - f(s, x(s), y(s), z(s),u(s))]Ie, (s)}ds (3-7) 

- z(s)dW(s). 

In the following theorem, we will prove the above assumption (13.61) . 
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(3.8) 


Theorem 3.4 Suppose (Al) and (A2) hold. Then, for any P >2, 

E[ sup \fit) - + r \^%t) - mfdt] = 0{e^), 

te[o,T] Jo 

E[ sup \f{t) - mf + ( r \zHt) - z{t)\"dt)^/^] = o{eP/^), (3.9) 

ie[0.T] Jo 

E[ sup \m\" + r \m\"dt] = 0(e2), (3.10) 

tG[0,T] Jo 

-E) sup \f{t)-y{t)-y{t)\‘^+[ \z^t) - z{t) - z{t)\'^dt] = o{e'^). (3.11) 

te[o.T] Jo 

Proof. We first prove (13.8p and (13.9p . Set 

h{t) = y^it)-y%t), hit) = zht) - r(t) - {p{t),6ait))lEAt), 
hit) = {pit),Sbit)) + {qit),Sait)) + \{Pit)5ait),5ait)), 

liit) = fit,xht),yht),zh't):uht)) - fit,xit)+xiit)+X2it),yht)+Iiit),zht)+hit),uit)), 
hit) = Safiihpit) + iP(t)5cr(t) + ^P'^ihSait), 

hit) = [Inxn,pit),<jf)it)pit) + qit)]. 

Then we can get 

yht) - yit) = o(e) + J^{his)lEAs) + his) + /y(s)(y^(s) - yis)) + his)izhs) - ^(s)) 

+ [( 2 ;i(s) + ^ 2 ( 5 ))^, Ji(s),/ 2 (s)]Z) 2 /(s)[(a;i(s) + a; 2 (s))^,/i(s),/ 2 (s)]^ 

(3.12) 

+fzis){his),xiis))lEhs) - ^{his)D‘^ fis)llis)xiis), Xiis))}ds 
-ft^(^hs) - zis))dWis), 

where 

fit,xit) +xiit) +X2it),yht) + hit),zht) + hit),uit)) 

-fit,xit) +xiit) +X 2 it),yit) +Iiit),zit) +l 2 it),uit)) 

= fyis)iyhs) - yis)) + fhs)izhs) - zis)), 

/y(s) = lo + ^ih) + X2is), yis) + his) + fiiyhs) - yis)), 

zis) + his) + /i(^^(s) - zis)), uis))dpL, 

his) = fhs,xis) + xiis) + X2is),yis) + his) + nivhs) - yis)), 

z(s) + 12 ( 5 ) + At(z^(s) - zis)), uis))dp,, 

D'^fis) = fo fo ^(s) + V(a;i is) + X2is)), yis) + Xyhis), 

zis) + Xpthis),uis))dXdp. 
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Thus by Theorem 12.11 equation p.l2l) and standard estimates of BSDEs, we can easily obtain (13.81) and 
(|3.9I) . It is obviously for equation (13.101) . Now we prove (13.111) . Set 


F(t) = x^{t) - x{t) - Xi{t) - X2{t), y%t) = y^{t) - y{t) - y{t), z^t) = z^{t) - z{t) - z{t). 

By equations (13.121) and (13.7L we can get 

y‘^{t) = o(s) + ft^{fy(s)y^(s) + f^(s)z^(s) + (fy(s) - fy(s))y(s) + (f^(s) - f,is))z{s) 

+h{s) - [f{s,x{s),y{s),z{s) + (p{s),Sa{s)),u) - /(s)]/£;^(s) 

+ [(a;i(s) + X2(s))'^,/i(s),/2(s)]I)2/(s)[(a;i(s) + X 2 {s))'^, Ii{s), his)]'^ (3-13) 

+fz{s){l5{s),xi{s))lEAs) - ^{h{s)D^f{s)ll{s)xi{s),xi{s))}ds 
- z^s)dW{s). 

By Theorem 12.11 it is easy to check that we only need to show that 

E[iJo \{fy{s) - fyis))y{s) + (/^(s) - f^{s))z{s)\ds)‘^] = o{e^), 

E[{Jo \{l6{s){D^f{s) - ^D^f{s))li{s)xi{s),xi{s))\ds)^] = o{e^), 

Eiilo l^4(s) - [f{s,x{s),y{s),z{s) + {p{s),Sa{s)),u) - /(s)]/is,(s)Ms)2] = o{e‘^). 

Note that 

l/y(s) - fyis)\ + \fzis) - fzis)\ < C'(|a:i(s) + X 2 (s)| + |/i(s)| + |/ 2 (s)| + |j/®(s) - y(s)| + |z®(s) - z(s)|), 
and 

EKJo |<?(s)a;i(s)z(s)|ds)2] < £;[sup,g[o,T] ki(s)P(/o^ k(s)pds)(/o^ |z(s)pds)] 

< (i?[(/o" |xi(s)|8])V4(^;[(J^^ \q{s)\^dsy]y/^ 

= o(e^), 

(3.14) 

we can get E[(Jq |(/y(s) — fy{s))y{s) + (fzis) — / 2 (s))z(s)|c?s)^] = o(e^). Since D'^f is bounded, we can get 
that for each /3 > 2, 

E[{J^ \{D^f{s) - ^D^f{s))\\q{s)\^dsf] -J> 0 as £ ^ 0. 

Thus we can easily deduce E[{Jq \{Iq{s){D‘^ f{s) — ^Il^/(s))/J(s)a;i(s),xi(s))|ds)^] = o{e^). It is easy to 
verify that 

l-f4(s) - [f{s,x{s),y{s),z{s) + {p{s),Sa{s)),u) - f{s)]lE,{s)\ 

< C{|i®(s)| + [|xi(s) +X 2 (s)| + \y‘^(t) - y(t)\ + |r(s) - z(s)| + |Ji(s)| + |/ 2 (s)|]/is,(s)}. 










Since 


E[{ f \q{s)xi{s)\lE,{s)dsf]<E[sup \xi{s)\'^ [ |g(s)pds]e 

Jo se[o,T] Je^ 

<iE[snp \x,{s)\^]r/\E[{ [ \q{s)fdsr]f/^e 

se[o,r] JEe 

= o{e^), 

we can obtain E[{J^ 1^4(5) ~ [/(^i x^s), y{s),z{s) + {p{s),da{s)),u) — f{s)]lE^{s)\ds)'^] = o(e^). The proof is 
complete. □ 

Thus we obtain the following variational equation for BSDE dLSl): 

y^{t) = y{t) + {p{t),xi{t) + X2{t)) + ^{P{t)xi{t),xi{t)) +y{t)+o{e), 
z^{t) = z{t) + {p{t), 6 a{t))lE,{t) + {<xl{t)p{t) E q{t),xi{t) P X2{t)) 

(3.15) 

+ {dal{t)p{t) + \P{t)5<7{t) + \P'^{t)6a{t),xi{t))lE,{i) 

+ E P{t)^x{t) + <T'l{t)P{t) + Q{t)\xi{t),xi{t)) + z{t) + o{e). 

Remark 3.5 ITe can also give the variational equations for BSDE il.3\) as in Qy . Set 

yi(.t) = {p{t):Xi{t)), Zi{t) = (p(t),(5cr(t))/£;^(t) + (crj(<)p(<)+(7(t),a;i(t)), (3.16) 

it is easy to check that (yi,zi) satisfies the following BSDE: 

/ 

-dyi(t) = {{fx(t),xi(t)) + fy{t)yi{t) + f^(t)zi{t) 

' -[fzit){pit),d(x{t)) + {q{t),da{t))]lEAt)}dt- zi{t)dW{t), (3-17) 

j/i(T)= (Mx{T)),xiiT)). 

Set 

y2{t) = {p{t),X2{t)) + ^{P{t)xi{t),xi{t)) +y{t), 

Z 2 {t) = {al{t)p{t) +q{t),X 2 {t)) + {5al{t)p{t) + \P{t)5a{t) + ^P’^ {t)Sa{t),xi{t))lE,{t) (3-18) 

+ ^{[^Lit)pit) + P(t)<^xit) + a'^{t)Pit) + Qit)]xi{t),xi{t)) + z{t), 

it is easy to verify that 

/ 

-dy2{t) = {{fx{t),X2{f)) P fy{t)y2{t) + fz{t)z2{t) 

+ i[(cri(t))^, y^{t),z^{h)]D^f{f)[{^^{tW, 2/1W, zii^V 
' +\{y{t),da{t)) - \ f:,^{t){{p(t),5a{t))f + f{t, x{t), y(t),z{t) + {p{t),5a{t)),u) ( 3 - 19 ) 

-fit, xit), y{t),zit),u{t))]lE, it) + {Lit), xi(t))/£;, it)}dt - Z 2 it)dW it), 
y2iT)= {cfxixiT)),X2iT)) + \{cfxxixiT))x^{T),x^{T)), 
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where {L{t),xi(t))lE^{t) = o(e), so we do not give the explicit formula for L{t). Here we use 


[f{t,x{t),y{t),z{t) + {p{t), 6 a{t)),u) - f{t,x{t),y{t), z{t),u{t))]lE,{t) 

to completely deal with the term (p{t), Sa(t))lEsit) in the variation of z, so the terms fz(t){p{t),5(j(t)) and 
\fzz{t){{p{t)T5<7{t)))'^ are repeated in fz{t)zi{t) and ^fzz{t){zi{t))^. Noting equations iS. 1 6\} and iS.18\) . 
then the adjoint equations for {zi{t))'^ and other terms are essentially for xi{t), X 2 {t) and xi{t){xi{t))'^, 
which is solved in 0y. In order to further explain the difference of expansions for SDE and BSDE, we 
consider the following equations: 

I -dyi(t) = {{fz:{t),xi(t)) + fy{t)yi{t) + fz{t)zi{t)}dt-ii{t)dW(t), 

\ (3.20) 

[ yi{T)= {(j)x{x{T)),xi{T)), 

/ 

-dy 2 {t) = {{fx{t),X 2 {f)) + fy{t)y2{t) + fz{t)z2{t) 

' +[/(<, x{t), y{t),z{t) + {p{t),Sa{t)),u) - f{t, x{t), y{t),z{t),u{t)) (3-21) 

-fz{t){p{t),5a{t)) - \ fzz{t){{p{t),5a{t))f]lEM}dt - Z2{t)dW{t), 

UT) = {cfx{x{T)),X2{T)) + \{dqxx{x{T))x^{T),x^{T)). 

By the standard estimates of BSDEs, it is easy to show that 

E[ sup \yi{t) + y 2 {t) - yi{t) - y 2 {t)\‘^ + [ \zi{t) + Z 2 {t) - zi{t) - Z 2 {t)\‘^dt] = o(e^). 
tG[0,T] Jo 

Thus by equation Vt.l5\) . we can get 


y^{t) = y{t) + yi{t) + y 2 {f) + o{e), 
z^ff) = z{f) + zi{f) + Z 2 {t) + o{e). 


The main difference is equation \S.21\) which is due to the term (p(t),6a(t))lE„(t) in the variation of z. If 
f is independent of z, the variational equations for (y, z) are the same as in 3 /. which is pointed in Al . 


Now we consider the maximum principle. From equation (13.151) . we get 


J{u^{-)) - J{u{-)) = y^{0) - y{0) = y{0) + o{e). 
Define the following adjoint equation for BSDE dSH); 

I dj{t) = fy{t)-f{t)dt + fzit)-f{t)dW{t), 

7 ( 0 ) = 1 . 
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Applying Ito’s formula to we can obtain 

y(0) = E[J^ -f{s)[{p{s),6b{s)) + (g(s), (5cr(s)) + ^{P{s)Sa{s), Sa{s)) 

+/(s, x{s),y{s),z{s) + {p{s), 6 a{s)),u) - f{s, x{s), y{s), z{s),u{s))]Ie, {s)ds]. 
Note that 7 ( 5 ) > 0, then we define the following function: 


nit, X, y, z, u,p, q, P) = (p, 6 (t, x, u)) + {q, a{t, x, u)) 

+ ^(P(CT(t, x, u) — a{t, X, u)),a{t, X, u) — a{t, x, u)) (3.23) 

+f{t, x,y,z+ {p, a{t, X, u) - a{t, x, u)),u), 

where {p,q^P) is defined in equations (13.11) and (13.21) . Thus we obtain the following maximum principle. 

Theorem 3.6 Suppose (Al) and (A2) hold. Let u(-) be an optimal control and (x(-),y(-), z(-)) be the 
corresponding solution. Then 

n{t,x{t),y{t),z{t),u,p{t),q{t),P{t)) >n{t,x{t),y{t),z{t),u{t),p{t),q{t),P{t))yue U,a.e., a.s., (3.24) 

where %{■) is defined in IS. 2 A) . 

If the control domain U is convex, we can get the following corollary which is obtained by Peng in [l^ . 


Corollary 3.7 Let the assumptions as in Theorem \S.6\. If U is convex and b, a, f are continuously differ¬ 
entiable with respect to u, then 

{bl{t)p{t) + al{t)q{t) + f^{t)al{t)p{t) + fu{t),u - u{t)) > 0, Vit e U,a.e.,a.s.. 

Now we give an example to compare our result with the result in 


20 


24|. 


Example 3.8 Suppose n = d = k = 1. U is a given subset in M. Consider the following control system: 

dx(t) = u{f)dW{t), a:(0) = 0, 

y{t) = x(T)f{z{s))ds- z(s)dlT(s). 

In this case, our maximum principle is 

f{z(t) + M — u{t)) — fiz{t)) >0, Vm G U, a.e., a.s.. (3.25) 

Note that ^ ^ ^ 

y{t) — f u{s)dW{s)= [ f{z{.s) — u{s)u{s))ds — f {z{s) — u{s))dW{s), 

Jo J t J t 

then by comparison theorem of BSDE, it is easy to check that inequality iS. 251) is a sufficient condition. For 
the case U = {0,1}, /(O) = 0, /'(O) < 0, /(I) > 0, /(—I) < 0, it is easy to verify that {x, y, z, u) = (0,0, 0,0) 
satisfies (MW, thus u = 0 is an optimal control. But fz{z{t)){l — u(t)) < 0, which implies that u = 0 is not 
an optimal control for the case U = [0,1]. The maximum principle in ]2Al is fAz{t))(u — u{t)) > 0, Vu G U, 
a.e., a.s., which only cover the case U is convex. The maximum principle in l2(il contains two unknown 
parameters. 
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Remark 3.9 In 


2(i . \2A1 . the authors consider the control system which consists of SDE and the 

following state equation: 


y(t)=yo- f{s,xis),y{s),v{s),u{s))ds + v{s)dW{s), 
Jo Jo 

where the set of all admissible controls 


(3.26) 


W[0,T] = e W[0,r] X R X M2(0,r) : y{T) = <f{x{T))}. 

The optimal control problem is to minimize J(u{-),yo,v{-)) = yo orerW[0,T]. Obviously, this problem is 
equivalent to Peng’s problem. Thus our maximum principle also completely solves this control problem. 


3.2 Multi-dimensional case 

In this sebsection, we extend Peng’s problem to multi-dimensional case, i.e., the functions in BSDE (11.31) are 
m-dimensional, </>: R" —>• R™, / : [0, T] x R" x R™ x x R'” —>• R™. The cost functional is defined by 

J{u{-)) = h{y{0)), (3.27) 

where h : R”^ —>■ R. For deriving the variational equation for BSDE (II.3|) . we use the following notation. 


W{t) = iw\t),...,W‘^{t))^, f,(x) = ■■■,r^{x)Y, 

<j{t,x,u) = {a^^{t,x,u)), i = 1,... ,n,j = 1,... ,d, 

<J^{t,x,u) = {a^^{t,x,u),... ,a'’-\t,x,u)Y,j = 1 ,... ,d, 
fit, x, y, z, u) = {f^{t, x,y,z,u),..., f^(t, x, y, z, u))^, 
yit) = (y^it),---,y"'it)V,z{t) = {z^^it)),i < m,j < d, 
z^{t) = {z^i{t),...,z^\t))^,j = l,...,d. 

We introduce the following adjoint equations: for i = 1,..., m, 

I -dp,{t) = F,{t)dt - Yfj=i dl {t)dW^{t), 

[ p,{T) = 4J,{x{T)), 

j -dP,{t)= G,it)dt-j:‘j^,Qiit)dWHt), 

[ P,{T) = </)^,(x(r)), 

where Fi{t) and Gi{t) is given after the following notations: 

p{t) = [pi{t),...,Pm{t)]„xm, q^{t) = (<),... ,g^(t)]„xm, 

pi(.t) = iplit),---^p?{t)V, diit) = 

bLit)pi(.t) = J27=iPtit)(.Kx{t)V^ , 

= Er=i i = i,...,m, j = i,...,d. 


(3.28) 


(3.29) 


(3.30) 


(3.31) 
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F,{t) = +m + EZi +EUzmVQiit) 

+ EU EZi fZ + qHi)h 

cm = m)b^{t) + (6.(i))^p.(i )+YZi + (.^mVQm 

+fliAt)Qiit) + fliAt)i<^LiWPi{t)] + + {Inxn,p{t),{al{t)Ep{t) + qAt), ■. ■, 

{^AAVpA) + qAt)]D^fAt)[inxn,pit), i^AArpit) + qAt),..., {ai{t)Yp{t) + qAAA, 

(3.32) 

where is the Hessian matrix of /* with respect to (cc, y, ..., Let y{t) = (y^(t),..., y’”(t))'^, 

z{t) = {z^AA) be the solution of the following BSDE: 

vA) = St AyA)vA) + E'^j=ifzAsAAA + {p'^AAAA + E'^j=i[AAs)A^(^AA + \p'^{ s)5aAs)5aAs)] 

+/(s, x(s), y{s), z(s) + p'^(s)S(7(s),u) - f{s, x{s),y{s),z{s), u{A)}Ie, (s)]ds 

-EUI^ ^AAdwAs), 

(3.33) 

where 

P{t) = [Pi{t),... ,PmA)], P'^{s)6aAs)dcrAA = {{Piis)6aAs)AcrAA):---APm{s)d<^AAA<^AA)A- 
Similar to the analysis in Theorem l3.4l we can get the following variational principle: 


y^’AA = vAA + AiiA^^AA +X2A)) + 5(P*(t)a:i(t),a:i(t)) +yAA +o(e), 
Z^r,e(t) = PAA + {PrA)AA{A)lEAA + {{AAAVPriA + QAA.xAA + X2{A) 
+{A<^i{AA PtiA + \pAA5(t^ (A + lPAiAd(xAA,xAA)iEAA 
+UiAixiAApiiA + PAA^AA + Ai{AZP^{A + QAA]xAA,xAA) 

A) + o(e), i = 1 , ..., m, j = l,...,d- 

Let h G (^^(R™). Then we get 

J{uA-)) - J{A-)) = (^y(y( 0 )),i/( 0 )) + o(e). 

We introduce the following adjoint equation for BSDE ( 13 . 331 ) . 

I dAA = fSiAAAdt + eU flAAAAdwAA, 

I 7(0) = hy{y{ 0 )). 

Applying Ito’s formula to (7(t),y(t)), we can get the following maximum principle. 


(3.34) 


(3.35) 


13 




Theorem 3.10 Suppose (Al) and (A2) hold. Let u{-) be an optimal control and (x(-), j/(-), z(-)) be the 
corresponding solution. The cost function is defined in ^S.27^ and h S C^(]R™). Then 

m,p^{t)6b{t)+j:‘^.^,[{qHt)rSa^{t) + ^P^{t)Sa^{t)Sa^{t)] 

+fit,xit),y{t),z{t) +p'^{t)da{t),u) - f{s,x{s),y{s),z(s),u{s))) (3-36) 

>0, Vii G U, a.e., a.s., 

where p, , P, 7 are given in equations iS.29\} . iS. 30d . \3.3‘A) and i3. 35\) . 

4 Problem with state constraint 

For the simplicity of presentation, suppose d = m = 1, the multi-dimensional case can be treated with the 
same method. 

We consider the control system: SDE (11.111 and BSDE (11.311 . The cost function J{u{-)) is defined in (11.411 . 
In addition, we consider the following state constraint: 

E[:^(x(T),2/(0))] =0, (4.1) 

where ip : K." x R —>■ K. We need the following assumption: 

(A3) p is twice continuously differentiable with respect to (x,?/); D'^p is bounded; Dp is bounded by 
C{l + \x\ + |y|). 

Define all admissible controls as follows: 

T] = {u(-) G W[0, T] : E[p{x{T),y{Q))] = 0}. 

The control problem is to minimize J{u{-)) over Wod[0,r]. 

Let u{-) G Had[0, T] be an optimal control and (x(-), y{-), z(-)) be the corresponding solution of equations 
dm and (|1.3I1 . Similarly, we define {x{-),y{-), z{-),u{-)) for any u{-) G Uf),T\. For any p > 0, define the 
following cost functional on W[0,T]: 

JpH-)) = {[(j/(0) - m) + + \E[p{xlTl 2/(0))]p}'/^. (4.2) 

It is easy to check that 

I Jp(M(-))>0, V?r(-)GW[0,T], 

I Jpi.u(,-)) = p< inf„GW[o.T] Jpiui-)) + p. 

In order to use well-known Ekeland’s variational principle, we define the following metric on W[0,T]: 

=E[I I{.^^^}{t,uj)dt]. 

^0 
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Suppose that (W[0,T],d) is a complete space and Jp(-) is continuous, otherwise we can use the technique in 


20j and the result is the same. Thus, by Ekeland’s variational principle, there exists a Mp(-) G W[0,T] 
such that 


Jpiupi-)) < p, d{up{-),u{-)) < y/p, 

Jp{u{-)) - Jp{up{-)) + y/pd(up{-),u{-)) > 0, Vu(-) G U[0,T]. 
For any e > 0, let C [0,T] with |Fe| = e, define 


(4.3) 


“p(^) = Up{t)lE^{t) + ulE,{t), Vm G U. 

It is easy to check that d{up{-),Up{-)) < s. Let {xp{-),yp{-), Zp{- j) be the solution corresponding to Up{-). 
Similarly for ixp{-),yp{-), Thus by (lOl) , we can get 


0 < Jp{u^p{-)) - Jp{up{-)) + y/pe 

< ^p[2/p(0) - yp(0)] + Mp{£’b(a;^('r), 2/p(0))] - E[ip{xp{T),yp{0))]} + yfpe + 0(e), 


(4.4) 


where 

Ap = Jp(Mp(-))”M(2/p(0) - j/(0)) +p], p-p = Jp{up{-))-^E[ip{xp{T),yp{Q))]. 

The same analysis as in Theorem l3.41 let {pP {■), qP {■)) and {PP{-), Qp{-)) be respectively the solutions of equa¬ 
tions (13.1|) and (13.21) with (a;(-), !/(•), z(-), u{-)) replaced by {xp{-), yp{-), Zp{-), Up{-)), and let all the coefficients 
be added by a superscript p. Then 

2/p(0)-2/p(0) = yp(0)-fo(e), (4.5) 

where 


ypW = I^{fvis)yp{s) + fPis)zp{s) + [{pPis),SbP{s)) + {qP{s),6aP{s)) + i(P^(s)dcr'’(s), dcr^(s)) 

+/(s, Xpis),yp{s), Zp{s) + {pPis), 6aP{s)),u) - f{s, Xpis),yp{s), Zp{s), Upis))]lE, (s)}ds (4-6) 
-^pis)dW{s). 

Similarly, let 

I -dpg(t)= [(6g(t))^pg(t) + (a£(t))^go^(t)]dt-<Zo^(t)dIT(t), 

[ Po{T)= ppp^{xpiT),yp{0)), 

-dpP{t) = [{bP^{t)Y' Po{t) + PS{t)hP^{t) + {aP{t)fP^it)aP{t) + {aP{t)fQ^it) + Q^{t)aP{t) 

+ ibLit)rP^oit) + i<yLit)rQmdt - Q^oit)dW{t), 

Po{T) = ppPa:xixp{T),yp{0)). 

Then 

Pp{E[p{x^p{T), 1/^(0))] - E[p{xp{T),yp{Q))]} 

= E[Jq {(p^Q{s),5bP{s)) + (ql^(s),S(7P(s)} + ^(Pa^(s)SaP(s),S(7P(s)}}lEAs)dsj (4-7) 

+ Pp^l¥’y(xp(T),yp(0))]yp(0) + o(s). 
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Define the following adjoint equation for BSDE (14.61) : 

I dY{t) = fP{t)Y{t)dt + fp(t)Y{t)dW{t), 

[ = Ap + ^ipE[ipy{xp{T),yp{Q))]. 

Then we can get 

{Ap + //p£;[(/3y (a:p(r), yp(0))]}yp(0) 

= Y{s){{p^{s),5hP{s)) + {qP{s), 5aP{s)) + \{PP{s)5aP{s), 5aP{s)) ( 4 - 8 ) 

+ /(s, Xp{s), yp{s),Zp{s) + {pP{s), 6aP{s)),u) - f{s, Xp{s), yp{s), Zp{s),Up{s))}lE, (s)ds]. 

Define the following function: 

nit, X, y, z, u, x', u',po, qo, Pq,p, q, P, 7 ) 

= {Po + IP, bit, X, u)) + {qo + ■yq, a{t, x, u)) 

(4.9) 

+ ^{{Po + yP){(T{t,x,u) - a{t,x',u')),a{t,x,u) - a{t,x',u')) 

+ lit) fit, x,y,z+ {p, (T{t, X, u) - a{t, x',u')),u). 

It follows from (14.41) . (14.51) . (14.7p and (14.81) that 

0<E[f {n{t, Xp{t),yp{t), Zp{t), u, Xp{t), Up{t),p^{t), qP{t), P^(t),pP(t), qP{t), PP(t), yP{t)) 

Jo 

-nit^Xp{t), yp(t ), Zp{t), Up(t), Xp(t), Up(t),p^(t), q({{t), P^(t), pP(t), qP(t ), pP(t), Y{t))}lE, {t)dt] 

+ y/pe + 0(e). 

Thus we obtain 

nit,Xp{t),yp{t),Zpit),u,Xp{t),Upit),p^{t),qYt),PYt),P^{t),qP{t),P^{t),Yit)) 

>n{t,Xp{t),yp{t),Zp{t),Up{t),Xp{t),Up{t),p^{t),qYt),Po{t),pP{t),qP{t),P''{t),yP{t)) 

— y/p, Vm S U, a.e., a.s.. 

Obviously, |App + |^pp = 1. Thus there exists a subsequence of {Xp,pp) which converges to {X, p) with 
|Ap + ImP = 1 as p —>■ 0. Note that d{up{-), u{-)) < ^P, then we can get for further subsequence 

(:rp(.), 2/p(-), «p(-),pS(-), 9g(-), 7^-)) ^ 

ix{-),y{-),z{-),u{-),po{-),qoi-),Po{-),p{-),qi-),Pi-),'f{-)), a.e., a.s., 
where {p{-),q{-)) and {P{-),Q{-)) are respectively the solutions of equations (13.ip and (13.2L 

-dpo{t)= [{Y{t)Ypo{t) + {aYt)Yqo{t)]dt-qo{t)dW{t), 

\ (4.10) 

Po{T)= ppYxiY.Y'b)), 
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(4.11) 


/ 

-dPoit) = [{b^(t))'^ Po{t) + Po{t)bx{t) + Po{t)aa:it) + Qo(t) + Qo(t)aa:(t) 

+iKxit))'^Po{t) + iaa;xit))'^qoit)]dt - Qo{t)dW{t), 

Po(T)= fiip^^{x(T),y{0)), 


I dj(t) = fy{t)j{t)dt + f^(t)-f{t)dW{t), 

I j{0) = X +yE[(py{x{T),y{0))]. 

Thus we get the following theorem. 


(4.12) 


Theorem 4.1 Suppose (Al), (A2) and (AS) hold. Letu{-) be an optimal control with state constraint 
and (x{-),y(-), z(-)) be the corresponding solution. Then there exist two contants X, y with |Ap + |/xp = 1 
such that 

n{t, x{t),y{t), z{t), u, x{t),u{t),po{t),qo{t), Po{t),p{t),q{t), P{t), j{t)) 

> n{t, x{t),y{t), z{t), u, x{t),u{t),po{t), qoit), Poit),p{t),q{t), P{t), 7 (t)), 

Vu G U, a.e., a.s., 

whereTLi-), ip{-),qi-)), {P{-),Q{-)), {po{-),qo()), (Po(-)> Oo(-)) and-/{■) are defined in fO]), (ED, 

lETp , UUD and (E7D - 
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